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CAMERA AND METHOD OF PHOTOGRAPHING GOOD IMAGE 

This application is a Divisional of co-pending 
Application No. 09/586,600, filed on June 2, 2000, the 
5 entire contents of which are hereby incorporated by 
reference and for which priority is claimed under 3 5 U.S.C. 
§ 120; this application also claims priority based on a 
Japanese patent applications, Hei 11-157159 filed on June 
03, 1999, and Hei 11-158666 filed on June 04, 1999, the 
10 contents of which are incorporated herein by reference. 

BACKGROUND OF THE INVENTION 

1. Field of the Invention 

15 

The present invention relates to a camera, and more 
particularly to a camera capable of automatically 
photographing an image of a subject when the subject 
satisfies a predetermined photographing condition. 

20 

2. Description of the Related Art 

Conventionally, a technique is known to correct a 
photograph so that a person photographed by a camera can be 
25 satisfied with the result. However, this technique requires 
a high degree of skill. Furthermore, it is difficult to 
correct a person's face in the photograph when he or she is 
blinking or is not smiling, to a face as if he or she is not 
blinking or is smiling. 

30 

On the other hand, Japanese Patent Laid-open 
Publication (Kokai) H9-212620 and Japanese Patent Laid-open 
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Publication (Kokai) H10-191216 disclose a technique to 
continuously photograph a plurality of images. Those images 
are displayed, and the person photographed by the camera can 
select a desirable image from among those images. 

5 

Japanese Patent Laid-open Publication (Kokai) H5- 
40303, H4-156526 and H5-100148 disclose cameras which can 
automatically judge the timing for photographing images. 

10 However, this was troublesome because the photographed 

person or the photographer needed to select the desired 
image by checking all of the images. Furthermore, when a 
lot of people are photographed in the image, it is more 
difficult to select an image that all of them are satisfied 

15 with. 

Furthermore, images are photographed at the timing 
when the photographer judges it is the best timing. 
Therefore, the photographers timing is not always matched 
2 0 with the best timing for the photographed person. In 
addition, when a lot of people are photographed in the 
image, it is more difficult to judge the best timing at 
which many of them will be satisfied with the image. 

2 5 SUMMARY OF THE INVENTION 

Therefore, it is an object of the present invention to 
provide a camera which overcomes the above issues in the 
related art. This object is achieved by combinations 

3 0 described in the independent claims. The dependent claims 

define further advantageous and exemplary combinations of 
the present invention. 
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According to the first aspect of the present 
invention, a camera comprises: an image data input unit 
forming an image of a subject for photographing said 
5 subject; a condition storing unit storing a predetermined 
photographing condition related to a desirable subject; and 
a timing signal generator outputting a timing signal when 
said subject satisfies said photographing condition. 

10 The camera may include an extractor extracting data of 

an aimed object from said image of said subject based on an 
extracting condition, wherein said photographing condition 
may include a predetermined condition related to a desirable 
aimed object and said timing signal generator outputs said 

15 timing signal when said aimed object satisfies said 
photographing condition . 

The extracting condition may be based on depth 
information of said image indicating the distance to each 
2 0 part of said subject. 

The extractor may detect data of a judgement location 
from said data of said aimed object in said image based on a 
detecting condition different from said extracting 
2 5 condition, said photographing condition may include a 
predetermined photographing condition related to a desirable 
judgement location, and the timing signal generator may 
output said timing signal when said judgement location 
satisfies said photographing condition. 

30 

The extractor may extract data of a plurality of said 
aimed objects from said image; and said timing signal 



FF-0016 



Attorney Docket No. 3562-0138P 

4 

generator may output said timing signal when said plurality 
of aimed objects satisfy said photographing condition. 

The timing signal generator may output said timing 
5 signal when the ratio of said aimed objects satisfying said 
photographing condition against all of said plurality of 
said aimed object exceeds a predetermined ratio. 

The extractor may detect data of a plurality of 
10 judgement locations from each of said data of said plurality 
of aimed objects based on a detecting condition different 
from said first condition, said photographing condition may 
include a predetermined photographing condition related to 
said judgement location, and said timing signal generator 
15 may output said timing signal when said plurality of said 
judgement locations satisfy said photographing condition. 

The timing signal generator may output said timing 
signal when the ratio of said judgement locations satisfying 

2 0 said photographing condition against all of said plurality 

of said aimed object exceeds a predetermined ratio. 

The camera may include an image -pickup control unit 
controlling said input unit for photographing said image 
25 based on said timing signal. 

The camera may include an illuminator illuminating 
said subject based on said timing signal. 

3 0 The camera may include a recording unit recording said 

image on a replaceable nonvolatile recording medium based on 
said timing signal. 
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The camera may include an alarm outputting an alarm 
signal for notifying that said subject satisfies said 
photographing condition based on said timing, signal. 

5 

The photographing condition may include a plurality of 
photographing conditions, and said camera may include a 
condition-setting unit previously selecting at least one of 
said photographing conditions, for photographing said image, 
10 from among said plurality of photographing conditions. 

The camera may include: an input condition determining 
unit determining an input condition for inputting said image 
based on information of said judgement location detected by 
15 said extractor; and an image- forming control unit 
controlling an input unit for forming said image of said 
subject based on said input condition. 

The camera as set forth in claim may include an image 
2 0 processing unit processing said image based on information 
of said judgement location detected by said extractor. 

According to the first aspect of the present 
invention, a camera comprises: an image data input unit 

25 forming a plurality of images of a subject for photographing 
said subject; a condition storing unit storing a 
predetermined photographing condition related to a desirable 
variation of said subject; a variation detector detecting 
variation of said subject in said plurality of said images 

30 based on information of said plurality of images; and a 
timing signal generator outputting a timing signal when said 
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variation of said subject satisfies said photographing 
condition. 

The camera may include: an extractor extracting data 
5 of an aimed object from each of said plurality of images of 
said subject based on an extracting condition, wherein said 
photographing condition may include a predetermined 
condition related to a desirable aimed object, said 
variation detector may detect variation of said aimed object 
10 in said plurality of images based on said information of 
said plurality of images, and said timing signal generator 
may output said timing signal when said variation of said 
aimed object satisfies said photographing condition. 

15 The extracting condition may be based on depth 

information of said plurality of images indicating the 
distance to each part of said subject. 

The extractor may detect data of a judgement location 
20 from said data of said aimed object in each of said 
plurality of images based on a detecting condition different 
from said extracting condition, said photographing condition 
may include a predetermined photographing condition related 
to a desirable judgement location, said variation detector 
25 may detect variation of said judgement location in said 
plurality of images based on said information of said 
plurality of images, and said timing signal generator may 
output said timing signal when said variation of said 
judgement location satisfies said photographing condition. 

30 

The photographing condition may include a 
predetermined starting condition for starting detection of 
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said variation of said judgement location, and said 
variation detector may start detecting said variation of 
said judgement location when said judgement location 
satisfies said starting condition. 

5 

The extractor may extract data of a plurality of said 
aimed objects from each of said plurality of images, said 
variation detector may detect variation of each of said 
plurality of said aimed objects in said plurality of images 
10 based on information of said plurality of images, and said 
timing signal generator may output said timing signal when 
said variation of said plurality of said aimed objects 
satisfy said photographing condition. 

15 The extractor may detect data of a plurality of 

judgement locations from each of said data of said plurality 
of aimed objects based on a detecting condition different 
from said extracting condition, said photographing condition 
may include a predetermined photographing condition related 

20 to desirable variation of said judgement location, said 
variation detector may detect variation of each of said 
plurality of said judgement locations in said plurality of 
images based on information of said plurality of images, and 
said timing signal generator may output said timing signal 

25 when said variation of said plurality of said judgement 
locations satisfy said photographing condition. 

The camera may include an image pickup control unit 
controlling said input unit for photographing said image 
30 based on said timing signal. 
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The camera may include an illuminator illuminating 
said subject based on said timing signal. 

The camera may include a recording unit recording said 
5 image on a replaceable nonvolatile recording medium based on 
said timing signal. 

The camera may include an alarm outputting an alarm 
signal for notifying that said subject satisfies said 
10 photographing condition based on said timing signal . 

The photographing condition may include a plurality of 
photographing conditions, and said camera may include a 
condition-setting unit previously selecting at least one of 
15 said photographing conditions for photographing said image, 
from among said plurality of photographing conditions. 

The timing signal generator may select said judgement 
location satisfying said photographing condition from among 
20 said plurality of said judgement locations in said plurality 
of images, and outputs information for said aimed object 
including said judgement location, and the camera may 
include: an input condition determining unit determining an 
input condition for inputting said image based on 

2 5 information for said judgement location; and an image 

forming control unit controlling an input unit for forming 
said image of said subject based on said input condition. 

The timing signal generator may select said judgement 

3 0 location satisfying said photographing condition from among 

said plurality of said judgement locations in said plurality 
of images, and outputs information for said aimed object 
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including said judgement location, and said camera may- 
include an image processing unit processing said image based 
on said information for said judgement location. 

5 According to the third aspect of the present 

invention, a method of photographing an image of a subject 
comprises outputting a timing signal when said subject 
satisfies a predetermined photographing condition. 

10 The method may include: extracting data of an aimed 

object from said image of said subject based on an 
extracting condition, wherein said photographing condition 
may include a predetermined condition related to a desirable 
aimed object, and said timing signal may be output when said 

15 aimed object satisfies said photographing condition. 

The extracting may include detecting data of a 
judgement location from said data of said aimed object in 
said image based on a detecting condition different from 
2 0 said extracting condition, said photographing condition may 
include a predetermined photographing condition related to a 
desirable judgement location, and said timing signal may be 
output when said judgement location satisfies said 
photographing condition . 

25 

The method may include photographing said subject 
based on said timing signal. 

The method may include recording said photographed 
30 image of said subject on a replaceable nonvolatile recording 
medium based on said timing signal. 
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The method may include: determining an input condition 
for inputting said image based on information for said 
judgement location detected in said detecting step; and 
forming said image of said subject based on said input 
5 condition. 

The method may include processing said image based on 
information for said judgement location detected in said 
detecting step. 

10 

According to the fourth aspect of the present 
invention, a method of photographing a plurality of images 
of a subject comprising: detecting variation of said subject 
in said plurality of said images based on information for 
15 said plurality of images; outputting a timing signal when 
said variation of said subject satisfies a predetermined 
photographing condition related to a desirable variation of 
said subject. 

2 0 The method may include extracting data of an aimed 

object from each of said plurality of images of said subject 
based on an extracting condition, said detecting may include 
detecting variation of said aimed object based on 
information for said image, and said timing signal may be 

25 output when said variation of said aimed object satisfies 
said photographing condition. 

The extraction of said aimed object may include 
detecting data of a judgement location from said data of 
30 said aimed object in each of said plurality of images based 
on a detecting condition different from said extracting 
condition, said detecting variation of said subject may 
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include detecting variation of said judgement location based 
on information for said image, and said timing signal may be 
output when said variation of said judgement location 
satisfies said photographing condition. 

5 

The photographing condition may include a 
predetermined starting condition for starting detection of 
said variation of said judgement location, and said 
detecting of variation may start detecting said variation of 
10 said judgement location when said judgement location 
satisfies said starting condition. 

The method may include photographing said image based 
on said timing signal. 

15 

This summary of the invention does not necessarily 
describe all necessary features so that the invention may 
also be a sub-combination of these described features. 

2 0 BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 shows a camera of the first embodiment 
according to the present invention, 

Fig. 2 is a block diagram of the control unit of the 
25 first embodiment, 

Fig. 3 is a block diagram of the function of the 
extractor, 

Fig. 4 is a flowchart showing a method of 
photographing an image, 

3 0 Fig. 5 is a flowchart showing in detail the method of 

extracting a face part, step 106 in Fig. 4, 
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Fig. 6 is a flowchart showing in detail the method of 
detecting a judgement location, step 108 in Fig. 4, 

Fig. 7 is a flowchart showing in detail the method of 
generating a timing signal, step 110 in Fig. 4, 
5 Fig. 8 is a flowchart showing in detail the method of 

photographing a refined image, step 112 in Fig. 4, 

Fig. 9 is a flowchart showing in detail the method of 
photographing a refined image, step 112 in Fig. 4, 

Fig . 10 is a flowchart showing in detail the method of 
10 generating a timing signal, step 110 in Fig. 4, 

Fig. 11 shows a camera of the second embodiment 
according to the present invention, 

Fig. 12 is a block diagram of the control unit of the 
second embodiment , 
15 Fig. 13 is a block diagram of the control unit of the 

third embodiment, 

Fig. 14 is a block diagram of the function of the 
extractor 60 , 

Fig. 15 is a block diagram of the function of the 

2 0 photographing condition judging unit, 

Fig. 16 is a flowchart showing in detail the method of 
detecting a judgement location, step 108 in Fig. 4, 

Fig. 17 is a flowchart showing in detail the method of 
generating a timing signal, step 110 in Fig. 4, 
25 Fig. 18 is a flowchart showing in detail the method of 

photographing a refined image, step 112 in Fig. 4, 

Fig. 19 is a flowchart showing in detail the method of 
generating a timing signal, step 110 in Fig. 4, 

Fig. 2 0 is a block diagram of the control unit of the 

3 0 fourth embodiment, 

Fig. 21 is a block diagram of the control unit of the 
fifth embodiment, 
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Fig. 22 is a flowchart showing a method of 
photographing an image, and 

Fig. 2 3 shows a camera of the sixth embodiment. 

5 DETAILED DESCRIPTION OF THE INVENTION 

The invention will now be described based on the 
preferred embodiments, which do not intend to limit the 
scope of the present invention, but exemplify the invention. 
10 All of the features and the combinations thereof described 
in the embodiment are not necessarily essential to the 
invention. 

Fig. 1 shows a camera 10 of the first embodiment 
15 according to the present invention. The camera 10 

continuously photographs raw images of a subject and 
determines the timing for photographing a refined image 
based on the previously photographed raw images. The camera 
10 photographs a refined image of the subject in accordance 
20 with the timing signal. Therefore, the timing for 

photographing a refined image may be automatically 
determined by the camera 10. 

The camera 10 includes an input unit 20, an A/D 
25 converter 30, a memory 40, a control unit 50, a release 
button 52, an alarm 54, a recording unit 90 and an output 
unit 92. The camera 10 of this embodiment further includes 
an illuminator 53. The camera 10 may be, for example, a 
digital still camera or a digital video camera that can 
3 0 photograph a still image. 
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The input unit 2 0 includes a parallactic image data 
input unit 22 and a normal image data input unit 24. The 
parallactic image data input unit 22 inputs parallactic 
images which are photographed from different viewpoints. 
5 The parallactic image data input unit 22 has a parallactic 
lens 32, a parallactic shutter 34, and a parallactic charge 
coupled device (CCD) 36. The parallactic lens 32 forms an 
image of a subject. The parallactic shutter 34 has a 
plurality of shutter units each of which serve as 

10 viewpoints. The parallactic shutter 34 opens one of the 
plurality of shutter units. The parallactic CCD 36 receives 
the image of the subject through the parallactic lens 32 and 
whichever of the shutter units of the parallactic shutter 34 
that are opened. The parallactic CCD 36 also receives 

15 another image of the subject through the parallactic lens 32 
and another of the shutter units of the parallactic shutter 
34, which is opened at this time. The images received 
through the parallactic lens 32 and the parallactic shutter 
34 form a parallactic image. Thus, the parallactic CCD 3 6 

2 0 receives the parallactic image of the subject formed by the 
parallactic lens 32 and converts it to electronic signals. 

The normal image data input unit 24 inputs a normal 
image photographed from a single viewpoint. The normal 

25 image data input unit 24 has a lens 25, a lens stop 26, a 
shutter 27, a color filter 28 and a charge coupled device 
(CCD) 29. The lens 25 forms an image of a subject. The 
lens stop 26 adjusts an aperture condition. The shutter 27 
adjusts exposure time. The color filter 28 separates RGB 

30 components of the light received through the lens 25. The 
CCD 2 9 receives the image of the subject formed by the lens 
25 and converts it to electric signals. 
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The A/D converter 30 receives analog signals from the 
parallactic image data input unit 22 and the normal image 
data input unit 24. The A/D converter 30 converts the 
5 received analog signals to digital signals and outputs the 
digital signals to the memory 40. The memory 40 stores the 
input digital signals. This means that the memory 40 stores 
the data of the parallactic image, the subject photographed 
by the parallactic image data input unit 22, and the data of 
10 the normal image of the subject photographed by the normal 
image data input unit 24. 

The control unit 50 outputs a timing signal for 
starting photographing of an image of a subject when the 

15 subject satisfies a predetermined photographic condition. 
The timing signal is input to the input unit 20. The camera 
10 then starts the photographing operation based on the 
timing signal, to obtain a refined image of the subject. 
The control unit 50 processes the photographed refined image 

20 and outputs the processed image. The control unit 50 
controls at least one of the following conditions: focus 
condition of the lens 25, aperture condition of the lens 
stop 26, exposure time of the shutter 27, output signal of 
the CCD 29, condition of the parallactic shutter 34, and 

25 output signal of the parallactic CCD 36. The control unit 
50 also controls the illuminator 53. 

The release button 52 outputs to the control unit 50 a 
signal for starting the photographing operation . This 
3 0 means that when a user of the camera 10 pushes the release 
button 52, the signal is output to the control unit 50. The 
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control unit 50 then controls the input unit 2 0 for 
photographing an image of the subject. 

As described above, the camera 10 is capable of 
5 automatically photographing a refined image of the subject 
by determining best timing for photographing the refined 
image. However, the camera 10 is also capable of 

photographing the image at a desirable timing for the user 
of the camera 10, when he/she pushes the release button 52. 
10 The camera 10 may have a switch, not shown in the drawings, 
for selecting an automatic photographing mode in which the 
best timing for photographing the image is automatically 
determined, and a manual photographing mode in which the 
user of the camera 10 determines the desirable timing. 

15 

The alarm 54 outputs an alarm signal upon receiving 
the timing signal from the control unit 50. The alarm 54 
may be, for example, an alarm generator or a light -emit ting 
diode. Thus, the user of the camera 10 can know the best 
2 0 timing determined by the camera 10 for photographing a 
refined image of the subject. 

The recording unit 90 records the image output from 
the control unit 50 on a recording medium. The recording 
2 5 medium may be, for example, a magnetic recording medium such 
as a floppy disk, or a nonvolatile memory such as a flash 
memory . 

The output unit 92 outputs the image recorded on the 
30 recording medium. The output unit 92 may be, for example, a 
printer or a monitor. The output unit 92 may be a small 
liquid crystal display (LCD) of the camera 10. In this 
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case, the user can see the image processed by the control 
unit 50 immediately after photographing the image. The 
output unit 92 may be an external monitor connected to the 
camera 10. 

5 

Fig. 2 is a block diagram of the control unit 50 
according to the first embodiment. The control unit 50 
includes an image pickup control unit 56, an image forming 
control unit 58, an extractor 60, a condition-storing unit 
10 70, a timing signal generator 80, an input condition 
determining unit 82, and an image processing unit 84. 

The extractor 60 receives a parallactic image 
photographed by the parallactic image data input unit 22 and 

15 a raw image photographed by the image data input unit 24, 
from the memory 40. The extractor 60 extracts an aimed 
object from the raw image based on the information obtained 
from the parallactic image and the raw image. The 
information includes image information of the raw image and 

20 depth information of the parallactic image. The aimed 
object defined here is an independent object at which a 
photographer aims when photographing. The aimed object may 
be, for example, a person in a room when the person and the 
objects in the room are photographed, a fish in an aquarium 

25 when the fish and the aquarium are photographed, or a bird 
stopping on a branch of a tree when the bird and the tree 
are photographed. 

The extractor 6 0 then detects a judgement location 
3 0 from the aimed object based on the information obtained from 
the parallactic images and the raw images. The judgement 
location defined here is a location to which specific 
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attention is paid when selecting a desirable image. The 
judgement location may be, for example, an eye of a person 
when the person is photographed, or a wing of a bird when 
the bird is photographed. The aimed object may be an area 
5 including the judgement location, extracted for a certain 
purpose. The information for the judgement location is 
output to the timing signal generator 80, the input - 
condition-determining unit 82 and the image-processing unit 
84 . 

10 

The condition-storing unit 70 stores predetermined 
conditions related to a judgement location which should be 
included in a raw image obtained by photographing a subject. 
The best timing for photographing a refined image of the 

15 subject in this embodiment is when the aimed object in the 
image is in good condition. This means that a judgement 
location included in the aimed object satisfies the 
predetermined conditions stored in the condition- storing 
unit 70. The condition- storing unit 70 may store a 

20 plurality of photographing conditions. The condition- 

storing unit 70 may include a condition-setting unit, not 
shown in the drawings, by which a user can select at least 
one of the photographing conditions from among a plurality 
of photographing conditions. 

25 

The timing signal generator 80 outputs a timing signal 
for photographing an image . The t iming s ignal generator 8 0 
outputs the timing signal when the judgement location 
detected by the extractor 60 satisfies the predetermined 
30 photographing condition stored in the storing unit 70. 
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The input -condition-determining unit 82 determines an 
input condition for inputting a refined image, based on the 
information for the aimed object or the judgement location 
received from the extractor 60. The input condition is 
5 output to the image-forming control unit 58. The input 
condition may be, for example, focus condition of the lens 
2 5 such that the aimed object including the judgement 
location is focussed. 

10 As the input unit 2 0 inputs an image in accordance 

with the input condition such as the focus condition of the 
lens 25, determined by the input-condition-determining unit 
82, the camera 10 can photograph a refined image in which 
the subject is in good condition. 

15 

The image-forming control unit 58 controls the input 
unit 2 0 to form a refined image of the subject based on the 
input condition determined by the input -condition- 
determining unit 82. This means that the image-forming 
20 control unit 58 controls at least one of the conditions 
including focus condition of the lens 25, aperture condition 
of the lens stop 26, exposure time of the shutter 27, and 
condition of the parallactic shutter 34, based on the input 
condition. 

25 

The image -pickup control unit 56 controls the input 
unit 2 0 to photograph a refined image of the subject based 
on the input condition determined by the condition- 
determining unit 70. This means that the image-pickup 
30 control unit 56 controls at least one of the conditions 
including output signal of the CCD 2 9 and output signal of 
the parallactic CCD 36, based on the input condition. The 
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output signal of the CCD 2 9 determines the gradation 
characteristics based on a gamma (y) curve and sensitivity. 

The image-pickup control unit 56 controls the input 
5 unit 20, to photograph a refined image based on the timing 

signal output from the timing signal generator 80. The - 

image-pickup control unit 56 controls the image -processing 
unit 84 to process the refined image. The image-pickup 
control unit 56 may control the illuminator 53, for flashing 
10 a light preceding or at the same time as photographing a 
refined image by the input unit 20. The image-pickup 
control unit 56 also controls the image-processing unit 84, 
to process the input refined image. 

The image-processing unit 84 receives the refined 
image photographed by the image data input unit 24 from the 
memory 40. The image-processing unit 84 then processes the 
refined image based on the information for the aimed object 
or the judgement location extracted from the extractor 60. 

Examples of the process condition for processing a 
normal image are explained in the following. 

The process condition for processing a normal image 
25. may relate to compression of the image. The process 

condition in this case is determined based on the data for 
the aimed object. The image-processing unit 84 separately 
determines the compressing condition of the image for the 
aimed object and for the components other than the aimed 
30 object so that the quality of the aimed object does not 
deteriorate, even though the data size of the image itself 
is compressed. The image processing unit 84 may separately 
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15 



20 



Attorney Docket No. 3562-0138P 
21 

determine the color compressing condition for the aimed 
object and the components other than the aimed object. 

The process condition for processing a normal image 
5 may relate to color of the image. The process condition in 
this case is determined based on the depth information. The 
processing-condition-determining unit 74 may, for example, 
separately determine the color condition for the aimed 
object and the components other than the aimed object, so 
10 that all the components have optimum gradation. 

The image -processing unit 84 may determine a 
processing condition in which the aimed object in the image 
is magnified and the magnified aimed object is composited 

15 with a background image. The background image may be the 
components included in the original image other than the 
aimed object, or an image previously selected by the user of 
the camera 10. The image -processing unit 84 may then 
composite the data for the aimed object and the data for the 

2 0 components other than the aimed object to form a composite 
image . 

As described above, the extractor 60 extracts the data 
for the aimed object and the judgement location from the 

2 5 image, and the aimed object and the judgement location can 

be processed separately from the components other than these 
parts . 

Since cameras are usually used to photograph human 

3 0 beings, the best timing for photographing a refined image 

means that the targeted person has a good appearance . The 
good appearance of the person may be when, for example, "the 
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person is not blinking", "the person's eyes are not red- 
eyed" , "the person is looking at the camera" , or "the person 
is smiling" . The condition-storing unit 70 stores these 
conditions as the photographing conditions. The condition- 
5 storing unit 70 may set a photographing condition by 
selecting at least one of the photographing conditions 
stored therein. 

The method of outputting a timing signal for 
10 photographing a refined image of a subject when a targeted 
person has a good appearance will be explained. The 
condition- storing unit 70 stores conditions such as "the 
person is not blinking", "the person's eyes are not red- 
eyed" , "the person is looking at the camera" , and "the 
15 person is smiling" as the photographing conditions. These 
photographing conditions relate to the face of the person, 
and more specifically to the eyes or mouth of the person. 
Therefore, it is assumed in this embodiment that the aimed 
object is the face area of the person and the judgement 
2 0 location is the eyes or mouth of the person. 

Each of the photographing conditions has a reference 
situation for the judgement location, which should meet the 
requirements of the photographing condition. The condition- 
25 storing unit 70 also stores the reference situations for the 
judgement location, each respectively corresponding to each 
of the photographing conditions. The reference situations 
for the judgement location corresponding to each of the 
photographing conditions will be described in the following. 

30 

For. the conditions such as "the person is not 
blinking", "the person's eyes are not red-eyed" and "the 
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person is looking at the camera" , the reference situation 
may relate to the shape of the eye , color of the eye , and 
size of the eye. For the condition such as "the person is 
smiling" , the reference situation may also relate to the 
5 size of the eye, as well as shape of the mouth, and size of 
the mouth. Whether each of the judgement locations 

satisfies each of these reference situations or not is 
judged in accordance with predetermined algorithms based on 
experience . 

10 

When the photographing condition "the person is not 
blinking" is selected, the judgement location may be the eye 
of the person. The reference situation for the eye in this 
photographing condition will be determined as follows. When 

15 a person blinks, his/her eyelid hides his/her eyeball. 
■While he/she is blinking and his/her eye is partially 
closed, the white part of his/her eyeball is especially 
hidden by his/her eyelid. This means that when the person 
is not blinking, the white part of his/her eyeball should be 

20 relatively large. Therefore, the reference situation for 
the photographing condition "the person is not blinking" 
becomes "the white part of his/her eyeball has a large 
dimension" . 

25 When the photographing condition "the person's eyes 

are not red-eyed" is selected, the judgement location may be 
the eyes of the person. The reference situation for the 
eyes in this photographing condition will be determined as 
follows. Eyes of a person are usually red-eyed when the 

3 0 person is photographed using a flash in a dark situation. 
This happens because the person's eyes cannot sensibly 
compensate for the sudden brightness and his/her pupils 
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become red. This means that when the person's eyes look 
red-eyed, his/her pupils in each iris become red and the 
rest of the iris does not become red. Typically, people of 
Asian descent have brown or dark brown colored irises, and 
5 people of European descent have green or blue colored 
irises. Therefore, the reference situation for the 

photographing condition "the person's eyes are not red-eyed" 
becomes "the red part in his/her iris has a small 
dimension" . 

10 

When the photographing condition "the person is 
looking at the camera" is selected, the judgement location 
may be the eye of the person. The reference situation for 
the eye in this photographing condition will be determined 
15 as follows. When a person is looking at the camera, a line 
between the camera and the iris of the person and a normal 
vector of his/her iris are almost the same. Therefore, the 
reference situation for the photographing condition "the 
person is looking at the camera" becomes "the normal vector 

2 0 of the iris in his/her eye is approximately equal to the 

angle of the line between the camera and his/her iris" . 

When the photographing condition "the person is 
smiling" is selected, the judgement location may be the eyes 
25 and the mouth of the person. The reference situation for 
the eyes and the mouth in this photographing condition will 
be determined as follows. When a person is smiling, 
although it depends on each person, his/her eyes become 
relatively thin. At this time, although it depends on each 

3 0 person, his/her mouth expands right-and-left wards and 

his/her teeth are shown. Therefore, the reference 

situations for the photographing condition "the person is 
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smiling" become "the white part in his/her eyes has a small 
dimension", "the width of his/her mouth is wide" and "the 
white area in his/her mouth has a large dimension" . 

5 Fig. 3 is a block diagram of the function of the 

extractor 60. The extractor 60 includes a depth 

information extractor 62, an image information extractor 64, 
an aimed object extractor 66 and a judgement location 
detector 68 . 

10 

The depth information extractor 62 extracts the depth 
information indicating the distance to each of components of 
the subject, based on the data for the parallactic image 
received from the memory 40. This means that the depth 

15 information extractor 62 determines a corresponding point 
for each of the components based on the parallactic image 
and gives a parallax amount. The depth information 

extractor 62 extracts the depth information based on the 
parallax amount of each of the components. Determining the 

2 0 corresponding point is a known technique, thus the 
explanation of this technique will be omitted. Extracting 
the depth information based on the parallax amount is also a 
known technique using the principle of triangulation, thus 
the explanation of this technique will be omitted. 

25 

The image information extractor 64 extracts the image 
information for normal images, from the data for the normal 
images received from the memory 40. The image information 
includes, for example, data for the normal image such as 
30 luminescence distribution, intensity distribution, color 
distribution, texture distribution, and motion distribution. 
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The aimed object extractor 66 extracts data for the 
face area of the person as the aimed object, based on the 
depth information and the image information. Each of the 
images may include, for example, a plurality of components. 
5 The aimed object extractor 66 recognizes each of the 
components based on the depth information. The aimed object 
extractor 66 then specifies the face area by referring to 
the depth information and the image information of each of 
the components. The method of specifying the face area will 
10 be described in the following. 

The aimed object extractor 66 receives the 
photographing condition from the condition-storing unit 70. 
The aimed object extractor 66 extracts the aimed object 

15 based on the photographing condition. In this embodiment, 
the aimed object is the face of the photographed person. 
Therefore, at first, the component including the face is 
specified depending on assumptions such as "the person 
should be close to the camera" , "the person should be in the 

20 middle of the image", or "the proportional relationship of 
the height of the person to the width and height of the 
image should be within a predetermined range" . The distance 
from the camera to each of the components in the image is 
evaluated based on the depth information. The distance from 

2 5 the center of the image to each of the components in the 
image, and the proportional relationship of the height of 
the components are evaluated based on the image information. 
Each of the values is multiplied by predetermined constants 
corresponding to each condition. The multiplied values are 

30 added for each of the components. The added values are 
defined as weighted averages. The component having the 
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largest weighted average is extracted as the component 
including the aimed object. 

The constants by which the values . for each of the 
5 components are multiplied may be predetermined based on the 
aimed object. In this embodiment, for example, the aimed 
object is assumed to be the face of the photographed person. 
Therefore, the aimed object extractor 66 specifies the area 
having a skin color as the face part, based on the image 

10 information. The colors of each of the components are 
evaluated based on the color distribution of the images. 
The values of the color distribution may also be multiplied 
by predetermined constants and the multiplied values are 
added for each of the components to give the weighted 

15 averages. 

As described above, the aimed object extractor 66 
extracts an aimed object based on the depth information in 
addition to the image information. Therefore, even when a 
2 0 plurality of people are photographed in the image and their 
faces are close to each other, the faces of the different 
people can be distinctly extracted. 

The judgement location detector 68 detects the 

2 5 judgement location from the data for the face area extracted 

by the aimed object extractor 66. The judgement location 
detector 68 receives the photographing condition from the 
condition-storing unit 70. The judgement location detector 
68 detects the judgement location based on the photographing 

3 0 condition. In this embodiment, the judgement location is 

the eyes or mouth of the photographed person. Therefore, 
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the judgement location detector 68 detects the eyes and 
mouth from the face area. 

There is relatively little variation in the eyes of 
5 people with respect to color, shapes or their place on the 
face. Therefore, patterns of eyes such as the color of the 
eyes, shape of the eyes, and the place of the eyes on the 
face are previously determined, and the parts which are 
approximately similar to the determined patterns of the eyes 

10 are recognized as the judgement location on the face. 
Similarly, there is relatively little variation in mouths of 
people with respect to color, shapes or place on the face. 
Therefore, patterns of the mouth are also previously 
determined and the parts which are approximately similar to 

15 the determined patterns of the mouth are recognized as the 
judgement location on the face. 

The extractor 60 detects the judgement location from 
the extracted aimed object based on the image information 
20 for the aimed object. Therefore, the extractor 60 does not 
extract locations having similar shapes to the judgement 
location from the subject other than the aimed object 
included in the image. 

25 The judgement location detector 68 then outputs the 

data for the detected judgement locations to the timing 
signal generator 80. 

Referring back to Fig. 2, the method for judging the 
3 0 best timing for photographing an image will be explained in 
the following. 
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The timing signal generator 80 receives the data for 
the detected judgement locations from the extractor 60. The 
timing signal generator 80 also receives the photographing 
condition from the condition-storing unit 70. The timing 
5 signal generator 80 compares each of the judgement locations 
based on the reference situation for the photographing 
condition. The timing signal generator 80 then generates a 
timing signal when the judgement location satisfies the 
reference situation for the photographing condition. 

10 

When the photographing condition "the person is not 
blinking" is selected, the judgement location is the eyes 
and the reference situation is "the white part of his/her 
eyeball has a large dimension", as described above. 

15 Therefore, the timing signal generator 80 calculates the 
dimension of the white part of the eye detected by the 
judgement location detector 68 for each of the images, based 
on the image information. The timing signal generator 80 
generates a timing signal when the dimension of the white 

2 0 part of the eye has a larger dimension than a predetermined 
dimension. The width of the eye is always the same, even 
when the person opens or closes his/her eye. Therefore, the 
predetermined dimension may be determined relative to the 
width of the eye. People usually blink both eyes at the 

25 same time, therefore, the timing signal generator 80 may 
check only one of the eyes of the photographed person. 
However, by checking both eyes, the desired judgement 
location can be selected more precisely. 

30 When the photographing condition "the person's eyes 

are not red-eyed" is selected, the judgement location is the 
eyes and the reference situation is "the red part in his/her 
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iris has a small dimension", as described above. Therefore, 
the timing signal generator 80 calculates the dimension of 
the red part in the iris of the eye detected by the 
judgement location detector 68 for the image, based on the 
5 image information. The iris of his/her eye is recognized as 
being a cylindrical or elliptic area whose circumference has 
a brownish or blue/green color. The timing signal generator 
8 0 generates a timing signal when the dimension of the red 
part of the eye has smaller dimension than a predetermined 
10 dimension. Both eyes of people are usually red eyed at the 
same time, therefore, the timing signal generator 8 0 may 
check only one of the eyes of the photographed person. 
However, by checking both of his/her eyes, the desired 
judgement location can be selected more precisely. 

15 

When the photographing condition "the person is 
looking at the camera" is selected, the judgement location 
is the eye and the reference situation is "the normal vector 
of the iris in his/her eye is approximately equal to the 

20 angle of the line between the camera and his/her iris", as 
described above. Therefore, the timing signal generator 80 
recognizes the iris as being a cylindrical or elliptic area 
whose circumference has a brownish or blue/green color. The 
timing signal generator 8 0 then recognizes the center of the 

25 iris and the normal vector of the center of the iris. The 
timing signal generator 80 generates a timing signal when 
the normal vector of the iris in the eye is closer to the 
line between the camera and the iris than a predetermined 
distance . 

30 

The normal vector of the iris can be obtained from the 
relative position of the camera and the face of the person, 
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the relative position of the face and the eyes of the 
person, and the relative position of the eyes and the irises 
of the person. The timing signal generator 80 may judge the 
desired judgement location based on the normal vector 
5 obtained from these relative positions. 

When the photographing condition "the person is 
smiling" is selected, the judgement location is the eyes or 
the mouth and the reference situation is "the white part in 

10 his/her eye has a small dimension" , "the width of his/her 
mouth is wide" or "the white part in his/her mouth has a 
large dimension", as described above. Therefore, the timing 
signal generator 80 calculates the dimension of the white 
part of the eye, the width of the mouth, and the dimension 

15 of the white part of the mouth detected by the judgement 
location detector 68 for each of the images, based on the 
image information. The timing signal generator 80 generates 
a timing signal when the white part of the eye has a smaller 
dimension than a predetermined dimension, when the mouth has 

2 0 a wider width than a predetermined width, or when the white 
part of the mouth has a larger dimension than a 
predetermined dimension. The predetermined dimension for 
the white part of the eye is relatively determined with 
respect to the width of the eye . The predetermined width 

25 for the mouth is relatively determined with respect to the 
width of the face of the person. The predetermined 

dimension for the white part of the mouth is relatively 
determined with respect to the dimension of the face of the 
person. 

30 

The timing signal generator 80 outputs a timing signal 
when the judgement location satisfies the above reference 
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situations. As described above, the control unit 50 

extracts the face part based on the raw image and the 
information for the raw image. The control unit 50 then 
detects the judgement location from the data for the 
5 extracted face part. As the camera 10 photographs a subject 
when the detected judgement location satisfies the 
photographing condition, the camera 10 can automatically 
photograph a desirable refined image without bothering the 
photographer . 

10 

The method of generating a timing signal when a 
plurality of people is photographed, will be explained next. 

When each of the images includes a plurality of people, 
15 the extractor 60 extracts the aimed object and detects the 
judgement locations for each of the people. This means that 
the aimed object extractor 66 extracts the face parts for 
each of the people from each of the images. The judgement 
location extractor 68 detects the eyes or the mouth for each 
20 of the people from each of the images. 

When each of the images includes a plurality of 
people, the timing signal generator 80 compares each of the 
judgement locations for each of the people based on the 

25 reference situation for the photographing condition. The 
timing signal generator 80 may generate a timing signal when 
the judgement locations for many of the people satisfy the 
reference situation for the photographing condition. The 
timing signal generator 8 0 may output the timing signal when 

30 the ratio of the judgement locations satisfying the 
photographing condition against all of the plurality of the 
judgement locations exceeds a predetermined ratio. In this 
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case, the camera 10 can photograph a refined image in which 
many of the people have a good appearance. 

Fig. 4 is a flowchart showing a method of photographing 
5 an image. The camera 10 starts photographing the subject 
when the release button 52 is pushed (S100) . When the 
camera 10 starts photographing, data for a parallactic image 
is input from the parallactic image data input unit 22 
(S102) . At the same time, data for raw images are 

10 continuously input from the image data input unit 24 (S104) . 
Then, the aimed object extractor 66 extracts the face part 
of the targeted person as the aimed object (S106) . The 
judgement location detector 68 detects the judgement 
location based on the image information for the face part 

15 (S108) . The timing signal generator 80 generates and 
outputs a timing signal when the judgement location 
satisfies a predetermined photographing condition (S110) . 
Upon receiving the timing signal, the image pickup control 
unit 56 controls the input unit 20 to photograph a refined 

20 image (S112) . 

The image-processing unit 84 processes the refined 
image, for example, compositing images and the like (S114). 
The recording unit 90 records the processed image on a 
2 5 recording medium (SI 16) . The output unit 92 outputs the 
recorded image (S118) . The photographing operation is 
terminated (S120) . 

Fig. 5 is a flowchart showing in detail the method of 

30 extracting a face part, step 106 in Fig. 4. The depth 

information extractor 62 extracts the depth information 

based on the parallactic image (S130) . The image 
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information extractor 64 extracts the image information 
based on the raw image (S132) . Then, the aimed object 
extractor 66 extracts the face part of the targeted person 
based on the depth information and the image information 
5 (S134) . When each of the images includes a plurality of 
people, the aimed object extractor 66 extracts the face 
parts for all of the people from each of the images (S136) . 

Fig. 6 is a flowchart showing in detail the method of 
10 detecting a judgement location, step 108 in Fig. 4. The 
judgement location detector 68 detects the judgement 
location based on the image information for the face part 
(S150) . When each of the images includes a plurality of 
people, the judgement location detector 68 detects the 
15 judgement locations for all of the people (S152 and S150) . 
Then, the input -condition-determining unit 82 determines the 
input condition based on the image information for the 
judgement location (S154) . 

20 Fig. 7 is a flowchart showing in detail the method of 

generating a timing signal, step 110 in Fig. 4. The timing 
signal generator 80 judges whether the judgement location 
detected by the judgement location detector 68 satisfies the 
photographing condition or not (S160) . The timing signal 

2 5 generator 8 0 continues judging whether the judgement 

location satisfies the photographing condition or not for a 
predetermined period (S164 and S160) . The timing signal 
generator 80 generates a timing signal when the judgement 
location satisfies the photographing condition (S162) . The 

3 0 image pickup control unit 56 controls the input unit 2 0 to 

stop photographing raw images when the judgement location 
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does not satisfy the predetermined photographing condition 
for a predetermined period (S164 and S166) . 

Fig. 8 is a flowchart showing in detail the method of 
5 photographing a refined image, step 112 in Fig. 4. The 
image pickup control unit 56 controls the input unit 2 0 to 
automatically photograph a refined image based on the timing 
signal output at the step 110 in Fig. 4 (S170) . The input 
unit 2 0 inputs the data for the refined image (S172) . 

10 

At the step 112 in Fig. 4, the camera 10 may not 
automatically photograph a refined image but the user of the 
camera 10 may push the release button 52 to photograph the 
refined image, upon receiving the alarm signal from the 
15 alarm 54. 

Fig. 9 is a flowchart showing in detail the method of 
photographing a refined image, step 112 in Fig. 4. The 
alarm 54 outputs an alarm signal such as an alarm sound or 
2 0 an alarm light based on the timing signal generated at the 
step 110 (S190) . When the user, or the photographer of the 
camera 10 notices the alarm signal, and then pushes the 
release button 52 (SI 92) , the camera 10 photographs a 
refined image (S194) . 

25 

As the alarm 54 outputs the alarm sound or the alarm 
light based on the timing signal, the user can photograph a 
refined image at an optimum timing, without having to judge 
the timing himself. Furthermore, the targeted person can 
30 also notice the timing by the alarm sound or the alarm 
light . 
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The alarm 54. may output an alarm signal such as an 
alarm sound or an alarm light when the timing signal is not 
output from the timing signal generator for a predetermined 
period . 

5 

Fig. 10 is a flowchart showing in detail the method of 
generating a timing signal in which the alarm 54 outputs the 
alarm signal, step 110 in Fig. 4. The timing signal 
generator 8 0 judges whether or not the judgement location 

10 detected by the judgement location detector 68 satisfies the 
photographing condition (S180) . The timing signal generator 
8 0 continues judging whether or not the judgement location 
satisfies the photographing condition for a predetermined 
period (S184 and S180) . The timing signal generator 80 

15 generates a timing signal when the judgement location 
satisfies the photographing condition (S182) . The alarm 54 
outputs an alarm signal such as the alarm sound and the 
alarm light when the timing signal generator 80 does not 
output the timing signal for a predetermined period (S184 

20 and S186) . The image pickup control unit 56 controls the 
input unit 2 0 to stop photographing raw images at this time 
(S188) . 

As the alarm 54 outputs an alarm signal such as an 
2 5 alarm sound and an alarm light when the timing signal is not 
output within a predetermined period, the photographer and 
the targeted person become aware of the fact that the 
targeted person does not meet the photographing condition, 
by the sound and the light. 

30 

Fig. 11 shows a camera 110 of the second embodiment 
according to the present invention. The camera 110 
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continuously photographs raw images of a subject. The 
camera 110 then photographs a refined image of the subject, 
in accordance with a predetermined input condition, at the 
timing when one of the previously photographed raw images 
5 satisfies a predetermined photographing condition. The 
camera 110 in this embodiment is a silver halide type camera 
by which an image of a subject is formed on a silver halide 
film. The camera 110 includes an input unit 120, an A/D 
converter30, a memory 40, a control unit 150, a release 
10 button 52 and an alarm 54. The A/D converter 30, the memory 
40, the release button 52 and the alarm 54 in this 
embodiment have the same structures and functions as those 
explained in the first embodiment. Therefore, the 

explanation of these parts will be omitted. 

15 

The input unit 120 includes a parallactic image data 
input unit 122, a raw image data input unit 124 and a 
refined image data input unit 130. The parallactic image 
data input unit 12 2 and the raw image data input unit 124 in 

2 0 this embodiment respectively have the same structures and 

functions as the parallactic image data input unit 22 and 
the image data input unit 24 explained in the first 
embodiment. The refined image data input unit 130 includes 
a lens 132, a lens stop 134, a shutter 136 and a 
25 photographing unit 138. The lens 132, the lens stop 134 and 
the shutter 136 in this embodiment respectively have the 
same structures and functions as the lens 25, the lens stop 
26 and the shutter 27 shown in Fig. 1 of the first 
embodiment. The photographing unit 138 receives an optical 

3 0 image of a subject and forms an image of the subject on a 

silver halide film. 
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The image data input unit 24 of the first embodiment 
inputs both a raw image and a refined image. As for the 
camera 110 of this embodiment, the raw image data input unit 
124 inputs an electronic raw image and the refined image 
5 data input unit 13 0 inputs a refined image and forms the 
refined image on a film. The raw image data input unit 124 
has a CCD for receiving the image of the subject in the same 
way as the data input unit 24 of the first embodiment. The 
raw image data input unit 124 outputs electronic signals for 
10 the image converted by the CCD. 

Fig. 12 is a block diagram of the control unit 150 
according to the second embodiment . The control unit 150 
includes an image pickup control unit 56 , an image forming 

15 control unit 58, an extractor 60, a condition- storing unit 
70, a timing signal generator 80 and an input -condition- 
determining unit 82. The extractor 60, the condition- 
storing unit 70, the timing signal generator 80 and the 
input -condition-determining unit 82 in this embodiment 

20 respectively have the same structures and functions as those 
of the first embodiment, thus the explanation of these parts 
will be omitted. 

The image-forming control unit 58 controls the input 

2 5 unit 12 0 to form an image of a subject. The image forming 

control unit 58 controls at least one of the following 
conditions of the input unit 120: focus condition of the 
lens 132, aperture condition of the lens stop 134 and 
exposure time of the shutter 13 6, based on the input 

3 0 condition determined by the input -condition-determining unit 

82. The image-pickup control unit 56 controls the . input 
unit 120 to photograph an image of a subject. The image- 
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pickup control unit 56 also controls the photographing unit 
138 to photograph a refined image, based on the input 
condition. 

5 In this embodiment, the camera 110 includes the raw 

image data input unit 124 for inputting an electronic raw 
image in addition to the image data input unit 13 0 for 
inputting a refined image. Therefore, the camera can 
automatically set an optimum condition for photographing a 
10 refined image of the subject. Thus, the desired refined 
image can be obtained without photographing a plurality of 
images using silver halide films, which can be expensive. 

A camera of the third embodiment according to the 
15 present invention will be explained in the following. The 
camera of this embodiment has the same structure as that of 
the first embodiment explained with reference to Fig. 1. 
The camera of the third embodiment continuously photographs 
raw images of a subject. The camera then photographs a 
2 0 refined image, in accordance with a predetermined input 
condition, at the timing when the previously photographed 
raw image satisfies a predetermined photographing condition. 
The camera 110 may have a switch, not shown in the drawings, 
for selecting an automatic photographing mode in which the 

2 5 best timing for photographing the image is automatically 

determined, and a manual photographing mode in which the 
user of the camera 110 determines the best timing. 

The camera of this embodiment has the same structure 

3 0 as that of the first embodiment and includes an input unit 

20, an A/D converter 30, a memory 40, a control unit 50, a 
release button 52, an alarm 54, a recording unit 90 and an 
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output unit 92. The camera of this embodiment may be, for 
example, a digital still camera or a digital video camera 
that can photograph a still image. 

5 Fig. 13 is a block diagram of the control unit 50 

according to the third embodiment. The control unit 50 
includes an image-pickup control unit 56, an image- forming 
control unit 58, an extractor 60, a condition- storing unit 
70, a photographing condition judging unit 80, an input- 
10 condition determining unit 82, and an image-processing unit 
84. 

The extractor 60 receives a parallactic image 
photographed by the parallactic image data input unit 22 and 

15 a normal image photographed by the image data input unit 24, 
from the memory 40. The normal image includes a raw image 
and a refined image. The extractor 60 extracts an aimed 
object from the normal image based on the information 
obtained from the parallactic image and the normal image. 

2 0 The information includes image information of the normal 
image and depth information of the parallactic image. The 
extractor 60 outputs data for the aimed object to the input- 
condition-determining unit 82 and to the image -processing 
unit 84 . 

25 

As described above, cameras are usually used to 
photograph human beings. Therefore, the best timing for 
photographing a refined image may be determined by the 
condition of a targeted person. Therefore, it is assumed 
30 that the extractor 60 extracts a face part of the targeted 
person as the aimed object in this embodiment. 
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The extractor 60 then detects a judgement location 
from the aimed object based on the information obtained from 
the parallactic images and the normal images. It is also 
assumed that the extractor 60 detects shapes or colors of 
5 the eyes or the mouth of the targeted person as the 
judgement location in this embodiment. 

The condition-storing unit 70 stores predetermined 
photographing conditions related to the judgement location, 

10 which should be included in each of the raw images obtained 
by photographing the subject. The condition-storing unit 70 
may store a plurality of photographing conditions. The 
condition- storing unit 70 may include a condition-setting 
unit, not shown in the drawings, by which a user can select 

15 at least one of the photographing conditions from among a 
plurality of photographing conditions. 

The best timing for photographing a refined image may 
be, for example, the timing when the targeted person does a 

2 0 predetermined motion. This means that the best timing may 
be the timing when the aimed object of the targeted person 
shows a predetermined variation. The predetermined 

variation may be, for example, "the person opens his/her 
eyes after he/she has been closing his/her eyes for more 

25 than two seconds" or "the person's vision of sight follows a 
predetermined trail" . The condition storing unit 70 stores 
these conditions as the photographing conditions. 

The photographing condition judging unit 8 0 outputs a 
30 timing signal for photographing an image. The photographing 
condition judging unit 80 outputs the timing signal when the 
judgement location detected by the extractor 6 0 shows a 
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predetermined motion that satisfies the predetermined 
photographing condition stored in the storing unit 70. 

The input-condition-determining unit 82 determines an 
5 input condition for inputting an image based on the 
information for an aimed object or the judgement location 
received from the extractor 60. The input -condition- 

determining unit 82 outputs the input condition to the image 
forming control unit 58. The input condition may be, for 

10 example, focus condition of the lens 25 such that the aimed 
object including the judgement location is focussed. As the 
input unit 2 0 inputs an image in accordance with the input 
condition such as the focus condition of the lens 25, 
determined by the input -condition-determining unit 82, the 

15 camera of this embodiment can photograph a refined image in 
which the subject is in good condition. 

The image- forming control unit 58 controls the input 
unit 2 0 to form a refined image of the subject based on the 

2 0 input condition determined by the condition-determining unit 
70. This means that the image-forming control unit 58 
controls at least one of the conditions including focus 
condition of the lens 25, aperture condition of the lens 
stop 26, exposure time of the shutter 27, and condition of 

25 the parallactic shutter 34, based on the input condition. 

The image pickup control unit 5 6 controls the input 
unit 20, to photograph a refined image of the subject based 
on the input condition determined by the condition- 
30 determining unit 70. This means that the image-pickup 
control unit 56 controls at least one of the conditions 
including output signal of the CCD 2 9 and output signal of 
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the parallactic CCD 36, based on the input condition. The 
image -pickup control unit 56 controls the input unit 20, to 
photograph a refined image based on the timing signal output 
from the photographing condition judging unit 80. The 
5 image-pickup control unit 56 controls the image -processing 
unit 84 to process the refined image. 

The image-processing unit 84 receives the refined 
image photographed by the image data input unit 24 from the 
10 memory 40. The image-processing unit 84 then processes the 
refined image based on the information for the aimed object 
or the judgement location extracted from the extractor 60. 
The refined image is processed in accordance with the 
process conditions as explained in the first embodiment. 

15 

Fig. 14 is a functional block diagram of the extractor 
60. The extractor 60 includes a depth information extractor 
62, an image information extractor 64, an aimed object 
extractor 66 and a judgement location detector 68. 

20 

The depth information extractor 62 extracts the depth 
information indicating the distance to each of components of 
the subject, based on the data of the parallactic image 
received from the memory 40. 

25 

The image information extractor 64 extracts the image 
information for normal images, from the data for the normal 
images received from the memory 40. The image information 
includes, for example, data of the normal image such as 
30 luminescence distribution, intensity distribution, color 
distribution, texture distribution, and motion distribution. 
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The aimed object extractor 66 extracts data for the 
face area of the person as the aimed object, based on the 
depth information and the image information. The aimed 
object is extracted in a similar manner as that explained in 
5 the first embodiment. 

The aimed object extractor 66 outputs the information 
for the aimed object to the input-condition-determining unit 
82 and the image-processing unit 84. 

10 

As described above, the aimed object extractor 66 
extracts an aimed object based on the depth information in 
addition to the image information. Therefore, even when a 
plurality of people are photographed in the image and their 
15 faces are close to each other, the faces of the different 
people can be distinctly extracted. 

The judgement location detector 68 detects the 
judgement location from the data for the aimed object 
20 extracted by the aimed object extractor 66. The judgement 
location is detected in accordance with a detecting 
condition different from the extracting condition for 
extracting the aimed object by the aimed object extractor 
66. In this embodiment, the judgement location is eyes or 

2 5 mouth of the photographed person. Therefore, the judgement 

location detector 68 detects the eyes and mouth from the 
face area. 

The judgement location detector 68 outputs the 

3 0 information for the judgement location to the photographing 

condition judging unit 80. 
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Fig. 15 is a block diagram of the function of the 
photographing condition judging unit 80. The photographing 
condition judging unit 80 includes a detection-starting unit 
85, a variation detector 86 and a judging unit 88. The 
5 photographing condition includes a predetermined 
photographing condition related to the motion of the 
judgement location of the aimed object, and the starting 
condition for starting detection of the motion of the 
judgement location . 

10 

The detection-starting unit 85 outputs a starting 
signal when the judgement location detected by the extractor 
60 satisfies a predetermined starting condition. The 
variation detector 86 starts detecting variation in the 

15 motion of the judgement location upon receiving the starting 
signal from the detection- starting unit 85. The judging 
unit 88 outputs the timing signal for photographing a 
refined image when the variation of the motion of the 
judgement location detected by the variation detector 86 

2 0 satisfies a predetermined photographing condition. 

The photographing conditions may be, for example, "the 
person opens his/her eyes after he/she has been closing 
his/her eyes for more than two seconds" or "the person's 
25 vision of sight follows a predetermined trail" . It is 
desirable that the photographing conditions are motions or 
variations of the targeted person, which the targeted 
person usually does not perform in front of the camera, in 
order to avoid misjudgment. 

30 

Each of the photographing conditions has a reference 
situation for the judgement location, which should meet the 
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requirements of the photographing condition. The condition- 
storing unit 70 also stores the reference situations for the 
judgement location, each respectively corresponding to each 
of the photographing conditions. The reference situations 
5 for the judgement location corresponding to each of the 
photographing conditions will be described in the following. 

For the conditions such as "the person is not 
blinking", and "the person is looking at the camera", the 
10 reference situation may relate to shape of the eye, color of 
the eye, and size of the eye. Whether each of the judgement 
locations satisfies each of these reference situations or 
not is judged in accordance with predetermined algorithms 
based on experience. 

15 

When the photographing condition "the person opens 
his/her eyes after he/she has been closing his/her eyes for 
more than two seconds" is selected, the judgement location 
may be the eye of the person. The reference situation for 

2 0 the eye in this photographing condition will be determined 
as follows. When a person blinks, his/her eyelid hides 
his/her eyeball. While he/she is blinking and his/her eye 
is partially closed, a white part of his/her eyeball is 
especially hidden by his/her eyelid. This means that when 

25 the person is blinking, the white part of his/her eyeball 
should be relatively small and when the person is not 
blinking, the white part of his/her eyeball should be 
relatively large. Therefore, whether the person opens 
his/her eyes or not is determined based on the dimension of 

30 the white part of his/her eyeball. 
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The starting condition for the photographing condition 
"the person opens his/her eyes after he/she has been closing 
his/her eyes for more than two seconds" becomes "the person 
closes his/her eyes" . The detection-starting unit 85 

outputs a starting signal when it detects the closed eye of 
the person. The variation detector 86 starts detecting 
variation of the eye upon receiving the starting signal. 
The variation detector 86 counts the period while the person 
keeps his/her eyes closed, from the data for the raw images 
continuously input. The variation detector 86 outputs the 
timing signal when the person opens his/her eyes after 
he/she has had his/her eyes closed for more than two 
seconds. It is desirable for the variation detector 86 to 
output the timing signal one second after the person opens 
his/her eyes rather than the moment when the person opens 
his/her eyes. 

When the photographing condition "the person's vision 
of sight follows a predetermined trail" is selected, the 
2 0 judgement location may be the eye of the person. The 
reference situation for the eye in this photographing 
condition will be determined as follows. The trail of the 
person's vision of sight can be detected by detecting the 
normal vector of the iris in the eye. At first, the iris of 

2 5 his/her eye is recognized, from his/her eye detected by the 

judgement location detector 68, as being a cylindrical or 
elliptic area whose circumference has a brownish or 
blue/green color. The center of the iris is then detected 
based on the image information for the eye. The normal 

3 0 vector of the center of the iris is obtained based on the 

depth information . 
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It is assumed in this photographing condition that the 
predetermined trail of the vision of sight is "the person 
looks upper left with respect to the camera, lower right 
with respect to the camera and then at the camera" , for 
5 example. The starting condition in this case becomes "the 
person looks to the upper left with respect to the camera" . 
The detection starting unit 85 outputs a starting signal 
when it detects that the person is looking to the upper left 
with respect to the camera. The variation detector 86 

10 starts detecting variation of the vision of sight of the 
person upon receiving the starting signal . The variation 
detector 8 6 detects the trail of the vision of sight based 
on the data for the plurality of input raw images. The 
judgement unit 88 outputs the timing signal when the trail 

15 is "upper left, lower right and then at the camera". 

The control unit 5 0 extracts the face part based on 
the data for the raw image and the information thereof and 
then detects the judgement location from the information for 

20 the extracted face part. The control unit 50 then detects 
the variation of the judgement location and determines the 
timing for photographing when the detected judgement 
location satisfies the photographing condition. Therefore, 
the camera of this embodiment can automatically photograph 

25 at a timing when the targeted person is in good condition. 

The method of generating a timing signal when a 
plurality of people is photographed will be explained next. 

30 When each of the images includes a plurality of 

people, the judgement location detector 68 detects the 
judgement locations for each of the people. This means that 
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the aimed object extractor 66 extracts the face parts for 
each of the people from each of the images. The judgement 
location extractor 68 detects the eyes or the mouth for each 
of the people from each of the images. 

5 

At this time, the variation detector 86 detects the 
variation of the judgement locations for each of the people. 
The judging unit 88 outputs the timing signal when the 
variation of the plurality of judgement locations satisfy 

10 the photographing condition. The judging unit 88 selects 
the aimed objects respectively, including the judgement 
locations whose variation satisfies the photographing 
condition. The judging unit 88 then outputs the information 
of the aimed objects including the selected judgement 

15 locations to the input -condition-determining unit 82 and the 
image-processing unit 84. 

The method of photographing an image in this embodiment 
is almost same as that of the first embodiment shown in 

2 0 Figs. 4 and 5. 

Fig. 16 is a flowchart showing in detail the method of 
detecting a judgement location, step 108 in Fig. 4. The 
judgement location detector 6 8 detects the judgement 
location based on the image information for the face part 
25 (S250) . When each of the images includes a plurality of 
people, the judgement location detector 68 detects the 
judgement locations for all of the people (S252 and S250) . 

Fig. 17 is a flowchart showing in detail the method of 

3 0 generating a timing signal, step 110 in Fig. 4. The 

detection starting unit 85 judges whether or not the 
judgement location detected by the judgement location 



FF-0016 



Attorney Docket No. 3562-0138P 
50 

detector 68 satisfies the starting condition (S260) . The 
detection-starting unit 85 continues judging whether or not 
the judgement location satisfies the starting condition for 
a predetermined period (S260 and S262) . The variation 
5 detector 86 starts detecting the variation of the judgement 
location when the judgement location satisfies the starting 
condition (S261) . The image-pickup control unit 56 controls 
the input unit 2 0 to stop photographing raw images when the 
judgement location does not satisfy the predetermined 
10 starting condition for a predetermined period (S262 and 
S263) . 

The judging unit 88 then judges whether the variation 
of the judgement location satisfies the photographing 

15 condition or not (S264) . The timing signal generator 80 
generates a timing signal when the variation of the 
judgement location satisfies the photographing condition 
(S2 65) . When the variation of the judgement location does 
not satisfy the photographing condition, the process returns 

2 0 to step S2 6 0 if the predetermined period is remaining. 
Then, the detection starting unit 85 judges again whether or 
not the judgement location detected by the judgement 
location detector 68 satisfies the starting condition 
(S260) . The image pickup control unit 56 controls the input 

2 5 unit 2 0 to stop photographing raw images when the 

predetermined period is expired (S266 and S267) . 

Fig. 18 is a flowchart showing in detail the method of 
photographing a refined image, step 112 in Fig. 4. The 

3 0 image-pickup control unit 56 controls the input unit 20 to 

automatically photograph a refined image based on the timing 
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signal output at the step 110 in Fig. 4 (S270) . The input 
unit 20 inputs the data for the refined image (S272) . 

At the step 112 in Fig. 4, the camera 10 may not 
5 automatically photograph a refined image but the user of the 
camera 10 may press the release button 52 to photographing 
the refined image upon receiving the alarm signal from the 
alarm 54. 

10 The method of manually photographing a refined image 

by the user of the camera 10 is in accordance with the 
flowchart shown in Fig . 9 , which is explained in the first 
embodiment. The alarm 54 outputs an alarm signal such as an 
alarm sound or an alarm light based on the timing signal 

15 generated at the step 110 (S190) . When the user, or the 
photographer of the camera 10 notices the alarm signal, and 
then he/she presses the release button 52 (S192) , the camera 
10 photographs a refined image (SI 94) . 

2 0 As the alarm 54 outputs the alarm sound or the alarm 

light based on the timing signal, the user can photograph a 
refined image at an optimum timing without having to judge 
the timing himself. Furthermore, the targeted person can 
also notice the timing because of the alarm sound or the 
25 alarm light. 

The alarm 54 may output an alarm signal such as an 
alarm sound or an alarm light when the timing signal is not 
output from the timing signal generator for a predetermined 

3 0 period. 
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Fig. 19 is a flowchart showing in detail the method of 
generating a timing signal in which the alarm 54 outputs the 
alarm signal, step 110 in Fig. 4. The detection-starting 
unit 85 judges whether or not the judgement location 
5 detected by the judgement location detector 68 satisfies the 
starting condition (S300) . The detection starting unit 85 
continues judging whether or not the judgement location 
satisfies the starting condition for a predetermined period 
(S300 and S304) . The variation detector 86 starts detecting 

10 the variation of the judgement location when the judgement 
location satisfies the starting condition (S302). The alarm 
54 outputs an alarm signal such as an alarm sound and an 
alarm light when the photographing condition judging unit 80 
does not output the timing signal for a predetermined period 

15 (S304 and S306) . Then, the image-pickup control unit 56 
controls the input unit 2 0 to stop photographing raw images, 
when the judgement location does not satisfy the 
predetermined starting condition for a predetermined period 
(S308) . 

20 

The judging unit 88 then judges whether or not the 
variation of the judgement location satisfies the 
photographing condition (S310) . The timing signal generator 
8 0 generates a timing signal when the variation of the 

25 judgement location satisfies the photographing condition 
(S312). When the variation of the judgement location does 
not satisfy the photographing condition, the process returns 
to the step S314 if the predetermined period is not 
remaining. Then, the detection starting unit 85 judges 

3 0 again whether or not the judgement location detected by the 
judgement location detector 68 satisfies the starting 
condition (S314 and S300) . At the step S314, the alarm 54 
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outputs an alarm signal such as an alarm sound and an alarm 
light when the predetermined period is expired (S316) . The 
image-pickup control unit 56 controls the input unit 2 0 to 
stop photographing raw images (S3 18) . 

5 

As the alarm 54 outputs the alarm signal such as the 
alarm sound and the alarm light when the timing signal is 
not output within a predetermined period, the photographer 
and the targeted person become aware of the fact that the 
10 targeted person does not meet the photographing condition 
because of the sound and the light. 

The camera of the fourth embodiment will be explained 
in the following. The camera of this embodiment is a silver 
15 halide type camera by which an image of a subject is formed 
on a silver halide film and has the same structure as that 
explained in the second embodiment shown in Fig. 11. 
Therefore, the explanation of the structure of the camera in 
this embodiment will be omitted. 

20 

Fig. 20 is a block diagram of the control unit 150 in 
this embodiment. The control unit 150 in this embodiment 
includes an image pickup control unit 56, an image forming 
control unit 58, an extractor 60, a condition storing unit 

25 70, a photographing condition judging unit 180, an input- 
condition-determining unit 82. The extractor 60, the 
condition storing unit 70, the photographing condition 
judging unit 180 and the input-condition-determining unit 82 
in this embodiment respectively have same the structures and 

3 0 functions as those explained in the first embodiment, 
therefore, the explanation of these parts will be omitted. 
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The image -forming control unit 58 controls the input 
unit 120 to form an image of a subject. The image- forming 
control unit 58 controls at least one of the following 
conditions of the input unit 120: focus condition of the 
5 lens 132, aperture condition of the lens stop 134 and 
exposure time of the shutter 136, based on the input 
condition determined by the input-condition-determining unit 
82 . The image pickup control unit 56 controls the input 
unit 12 0 to photograph an image of a subject. The image 
10 pickup control unit 56 also controls the photographing unit 
13 8 to photograph a refined image based on the input 
condition. 

In this embodiment, the camera 110 includes the raw 
15 image data input unit 124 for inputting an electronic raw 
image, in addition to the image data input unit 13 0 for 
inputting a refined image. Therefore, the camera can 
automatically set an optimum condition for photographing a 
refined image of the subj ect . Thus , a desired refined image 

2 0 can be obtained without photographing a plurality of images 

using silver halide films, which can be expensive. 

A camera of the fifth embodiment according to the 
present invention will be explained in the following. The 
25 camera of this embodiment continuously photographs images of 
a subject. The camera outputs a timing signal when the 
targeted subject in the image satisfies the photographing 
condition. Upon receiving the timing signal, the camera of 
this embodiment records one of the images, which was 

3 0 photographed at a predetermined earlier period than the 

timing signal, based on the timing signal, as a refined 
image . 
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The camera of this embodiment includes a control unit 
50. The structure of the camera of this embodiment other 
than the control unit 50 is the same as that explained in 
5 the first to fourth embodiments. Thus, the explanation of 
same parts will be omitted. 

Fig. 21 is a block diagram of the control unit 50 
according to the fourth embodiment. The control unit 50 

10 includes an extractor 60, a condition-storing unit 70, a 
timing signal generator 80, an image processing unit 84, and 
a image storing unit 140. The extractor 60, the condition- 
storing unit 70, the timing signal generator 80 and the 
image processing unit 84 are the same as those explained in 

15 the first to fourth embodiments. Although only the timing 
signal generator 80 is shown in Fig. 21, the part having the 
numeral 80 may be the photographing condition judging unit 
explained in the third and the fourth embodiments. 

20 The image storing unit 140 temporarily stores the 

images photographed by the image data input unit 24 and 
input from the memory 40. Each of the images is 

respectively stored with time records of when the image was 
photographed. The image storing unit 14 0 receives the 

25 timing signal from the timing signal generator 80 and then 
outputs one of the raw images photographed at a timing 
earlier than the timing signal by a predetermined period as 
the refined image, to the image processing unit 84. The 
image processing unit 84 processes the refined image based 

30 on the information for the extractor 60. 
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Fig. 22 is a flowchart showing a method of 
photographing an image. The camera starts photographing the 
subject when the release button 52 is pressed (S400) . When 
the camera starts photographing, data for a parallactic 
5 image is input from the parallactic image data input unit 22 
(S4 02) . At the same time, data for raw images are 
continuously input from the image data input unit 24 (S404) . 
The raw images are temporarily stored in the image storing 
unit 140. Then, the aimed object extractor 66 extracts the 
10 face part of the targeted person as the aimed object (S406) . 
The judgement location detector 68 detects the judgement 
location based on the image information for the face part 
(S408) . The photographing condition judging unit 180 

generates and outputs a timing signal when the judgement 
15 location satisfies a predetermined photographing condition 
(S410) . Upon receiving the timing signal, the image storing 
unit 140 selects one of the raw images photographed at a 
timing earlier than the timing signal by a predetermined 
period, as the refined image. The image storing unit 140 
2 0 outputs the refined image to the image -processing unit 84 
(S412) . 

The image-processing unit 84 processes the refined 
image (S414) . The processing of the refined image may 
25 include compositing a plurality of refined images and the 
like. The recording unit 90 records the processed image on 
a recording medium (S416) . The output unit 92 outputs the 
processed image (S418) , and the photographing operation is 
terminated (S420) . 
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The detailed operations of the steps 206, 208 and 210 
are the same as those explained in the previous embodiments. 
Thus, an explanation of these steps will be omitted. 

5 The image storing unit 14 0 may store all of the raw 

images which are photographed from a timing earlier than the 
timing signal by a predetermined period to the timing of the 
timing signal, as the refined images. In this case, the 
image-processing unit 84 processes the plurality of refined 
10 images. 

As described above, the camera stores the raw image 
which is photographed at a timing earlier than the timing 
signal by a predetermined period as the refined image, based 

15 on the timing signal. Therefore, the refined image is 
selected by considering the delay time, even when the 
extractor 60 takes a certain time for extracting the aimed 
object and detecting the judgement location. Thus, an image 
in which the targeted person has a good appearance can be 

20 obtained. 

Furthermore, the camera stores all of the raw images 
which are photographed from a timing earlier than the timing 
signal by a predetermined period to the timing of the timing 
25 signal, as the refined images. Therefore, an image in which 
the targeted person has a good appearance can be selected. 

Fig. 2 3 shows a camera 210 of the sixth embodiment 
according to the present invention. The camera 210 of this 
3 0 embodiment continuously photographs a plurality of raw 
images of a subject in the same way as the first to fifth 



FF-0016 



Attorney Docket No. 3562-0138P 
58 



embodiments. The camera 210 outputs a timing signal when 
the raw image satisfies the photographing condition. 

The camera 210 of this embodiment has the same 
5 structure as that of the first embodiment and further 
includes a communication unit 150. The camera 210 outputs 
the timing signal through the communication unit 150, to 
control operation of an external apparatus 160 based on the 
timing signal. The communication unit 150 of the camera 210 

10 sends the timing signal to the external apparatus 160 by a 
wireless means. The communication unit 150 of the camera 
210 and the external apparatus may be held in communication 
with each other by a wireless means such as via a radio or 
infrared radiation or by cables such as via a USB or a LAN. 

15 The external apparatus 160 may be, for example, a camera for 
photographing a refined image of the target, or an 
illuminator. 

In this embodiment, the camera 210 continuously 
20 photographs raw images of a subject. The camera 210 outputs 
a timing signal when the raw image satisfies a predetermined 
selecting condition. The timing signal is transferred from 
the camera 210 to the external apparatus 160 through the 
communication unit 150 of the camera 210. When the external 
25 apparatus 160 is another camera for photographing a refined 
image, the external apparatus photographs a refined image of 
the subject based on the timing signal from the camera 210. 

Using the camera 210 of this embodiment, even a silver 
3 0 halide type camera that does not include a raw image data 
input unit can photograph a refined image of a subject at 
the timing when the targeted person is in good condition. 
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Thus, a desired refined image can be obtained without 
photographing a plurality of images using silver halide 
films which can be expensive. 

5 As described above, according to the embodiments of the 

present invention, an image in which a targeted object 
satisfies a predetermined photographing condition. 

Although the present invention has been described by 
10 way of exemplary embodiments, it should be understood that 
many changes and substitutions may be made by those skilled 
in the art without departing from the spirit and the scope 
of the present invention which is defined only by the 
appended claims. 

15 
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